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Abstract. The main purpose of this article is to supplement the 
authors' results on degenerate principal series representations of 
real symplectic groups with the analogous results for metaplcctic 
groups. The basic theme, as in the previous case, is that their 
structures are anticipated by certain natural subrepresentations 
constructed from Howe correspondence. This supplement is nec- 
essary as these representations play a key role in understanding 
the basic structure of Howe correspondence (and its complications 
in the archimedean case), and their global counterparts play an 
equally essential part in the proof of Siegel-Weil formula and its 
generalizations (work of Kudla-Rallis). The full results in the meta- 
plectic case also shed light on the seeming peculiarities, when the 
results in the symplectic case are viewed in their isolation. 

1. INTRODUCTION: CLASSICAL INVARIANT THEORY AND ITS 
TRANSCENDENTAL ANALOG 

Let the complex orthogonal group O m act on the space of complex 
matrices M mn by matrix multiplication on the left: 

O m r\ M m ^ n . 

The First Fundamental Theorem (FFT) of classical invariant theory 
asserts that the ring of O m -invariant polynomials are generated by the 
fundamental invariants of degree 2: 

P(M mi „)° m =< r -| 1 < i, j < n >, 

where r^X) = Y!k=i x kiXkj, for X = (x^) G M m>n . 

Let S 2 (C n ) denote the space of complex n x n symmetric matrices 
and define 

Q : M m , n -> S 2 (C n ), 

X ^ x l x. 

Key words and phrases. Classical invariant theory, coinvariants, degenerate prin- 
cipal series, Howe correspondence. 
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Then FFT asserts that the pull-back map 

Q* : P(S 2 (C n )) -> P(M m , n ) 0ra 
is surjective. Alternatively, we have an affine embedding: 

i> : M m J/O m ^ S 2 (C n ), 

where M m>n //O m is the affine quotient of M m>n by O m . The Second 
Fundamental Theorem (SFT) of classical invariant theory is then a 
statement on the image of ip as an affine subvariety of S 2 (C n ). Note 
that both M m>n and S 2 (C n ) carry natural actions of GL(n, C), and all 
maps (Q, Q*, ip) are GL(n, C)-equivariant. 

We now state results of Kudla-Rallis [KRlJ and Lee-Zhu [LZ2J, which 
may be viewed as transcendental analogs of FFT and SFT alluded to 
above. 

Let 

0(p,q) r^M p+q , n {R), 

again by matrix multiplication on the left. Denote by G = Sp(2n,R) 
the real metaplectic group of rank n and P its Siegel parabolic sub- 
group. The rest of notation will be explained in Section [3J 

Theorem 1.1. (Kudla-Rallis) There is a natural topological embedding 
with closed image: 

1> M : S(M p+q>n (R)) OM ^ /»(:= Indg(;£)), 

as G -representations. 

Theorem 1.2. (Lee-Zhu) describes the image of ip p ^ q . 

In [LZ2] , the authors describe the image Q p,q of ip p ^ q when p + q is 
even, in which case the representations concerned factor through the 
linear group Sp(2n, R). In fact only the subcase of p — q = (mod 4) 
was treated in detail and the other subcase of p — q = 2 (mod 4) was 
left to the reader. 

The aim of the current article is to complete the description of Q p ' q 
(without any restriction on p and q). This is necessary and useful as 
the representations involved (the coinvariants) play a key role in un- 
derstanding the basic structure of Howe correspondence (through the 
doubling method |Hll IRaj IKuj ) and their global counterparts feature 
prominently in the proof of Siegel- Weil formula and its generalizations 
[KR3] . For a recent application of these results to first occurrence con- 
jecture of Kudla-Rallis, see [SZ] . As an additional benefit, this full 
description enables us to organize the statements in a more coherent 
way so that their structures emerge clearer to the reader. The basic 
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idea is very simple: since both Q p,g and I a {(j) are i^- multiplicity free 
(K being a maximal compact subgroup of G), and since the struc- 
ture of I a {<j) is known (Section [5) |L2j for the linear group), one can 
identify the image Q p ' q as a G-representation by knowing its i^-types. 
The fascinating point is that from this description, one concludes that 
the reducibilities of I a (u) are completely accounted for by the possible 
embeddings of Q p,q, s (in a precise way). We summarize this assertion 
in Theorem 16.11 which should be viewed as analogous to SFT (prover- 
bially stated) as "all relations among the fundamental invariants are 
generated by the obvious ones" . 

Here are some words on the organization of this article. In Section 
[2l we review the basics of Howe duality correspondence. In Section 
[3l we introduce the space of coinvariants, as a special case of Howe's 
construction of maximal quotients, and its embedding into the degen- 
erate principal series I a (u) of the metaplectic group. In Section [U we 
describe the transition coefficients of I a (a) and give its immediate con- 
sequences for irreducibility and complementary series. In Section we 
give the detailed structure of I a (c) at points of reducibility, which are 
again arrived by analyzing the transition coefficients. In Section [HI we 
complete our description of the image of ip P) g in I a (u). As proofs of 
(any new) results follow a similar line as those of |L2] or [LZ2J, we shall 
omit them. 

Finally we mention the following works which are closely related to 
the theme of this article: [KRU ILZT1 lLZ2l lLZ3l lYaj (for real groups) 
and |KR2| IKSl lYaj (for p-adic groups). 

2. Howe duality correspondence 

In this section, we review briefly Howe's theory of dual pair corre- 
spondence, for the case at hand. 

Let H = 0(p,q), which acts on M p+(?in (R). We have the dualised 
action of H on y = L 2 (M p+9jn (M)), the space of square integrable 
functions on M p+(Jin (R). Extending the action of H, there is a (unitary) 

representation of a metaplectic group Sp(2iV, R) (N = {p + q)n) on y, 
called the Schrodinger model of an oscillator representation. Here and 
after, for any real symplectic group Sp(2iV, R), Sp(2iV, R) denotes its 
metaplectic two fold cover. This representation depends on a choice of 
a non-trivial unitary character of R, which we fix once for all. Denote 
the corresponding oscillator representation by u>. 

The main feature of this set-up is the following: there is a reductive 
dual pair 

(H,G) = (0(p, 9 ),Sp(27i,R)) C Sp(2iV,R), 
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namely a pair of reductive subgroups in Sp(2iV, R) which are mutual 
centralizers. 

Let uj°° be the smooth representation of u> realized in y°°. For the 
case at hand, y°° = <S(M p+gin (R)), the Schwartz space of rapidly de- 
creasing functions on M p+9iTl (R). 

For any subgroup E of Sp(2iV, R), denote by E the preimage of 
E under the covering map Sp(2iV, R) — > Sp(2iV, R). If E is also re- 
ductive, denote by TZ(E, u>) the set of infinitesimal equivalent classes 
of irreducible admissible representations of E which are realizable as 
quotients by ^-invariant closed subspaces of y°°. 

Howe Duality Theorem ( |H2j ): 1Z(H ■ G, to) is the graph of a bijec- 
tion between 1Z(H,lu) and 1Z(G,u). 

This is the Howe quotient correspondence. To be more concrete, let 
p be an irreducible admissible representation of H, which is realizable 
as a quotient by a H- invariant closed subspace of y°° . Define f2(p) to 
be the maximal quotient of y°° on which H acts by a representation of 
class p. We have (isomorphism class to mean infinitesimal equivalent 
class): 

n(p) 0(p) 

where 0(p) is a G-module. 

Howe duality theorem asserts that 0(p) is a finitely generated ad- 
missible quasisimple representation of G, and has a unique irreducible 
G-quotient, denoted by 9(p): 

G(p) -» 6(p). 

The correspondence 

p h. 6{p) 

is the Howe quotient correspondence. 

3. The coinvariants and the embedding 
For the dual pair (H, G) = (0(p, q), Sp(2n,R)), let 

qp,q = H owe ' s maximal quotient corresponding to 
the trivial representation 1 of 0(p, q) 
= S '(M p+9in (R)) o(p,q) (the space of coinvariants). 

This is a representation of G. 
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Remark: the continuous dual of QP> q is S*(M p+qyn (R))°( p ' q \ the space 
of 0(p, g)-invariant tempered distributions on M p+9 n (IR). This is in- 
vestigated in [KR1] and |Zhj . 



From now on, we shift notation and will denote G = Sp(2n, R). 
The case of n = 1 differs from the general case slightly, but since it 
is straightforward, we omit it and will assume that n > 2 throughout 
this paper. We shall identify Sp(2n, R) set with 

Sp(2n,R) x Z 2 = {(g,e) : g G Sp(2n,R), e = ±1}. 

For a G GL(n, R) and b G M n (R) such that b = b\we let 

' a 
(a- 1 )* 



n b 



In b 



Let 
and 



In 

M = {(m a , e) : a G GL(n, R), £ = ±1} 



iV = {(n 6 ,l): 6 G M n (R), 6 = b 1 }. 
Then P = MiV is a maximal parabolic subgroup of G, called the Siegel 
parabolic. 

Let x '■ M — * C x be given by 

% if det a < 0, 



X(m a ,e) = £ ■ 



1 if det a > 0. 



This is a character of M and it is of order 4. For a = 0, 1,2,3 and 
a G C, let be the character of P given by 

x2[(m 0) £)Kl)] = \ det a\°x(rn a ,sr. (3.1) 

Let I a {&) be the normalized induced representation: 

J» = Ind£ X « 

The representation space of I a {<j) is 

{/ g C°°(G) : /(pp) = 8^{j>)x a Mf{g), Vg G G, p G P}, 
and G acts by right translation: 

g-f(h) = f(hg), (g,heG). 
Here 5 denotes the modular function of P, and is given by 

5[(m a ,e)(n b ,l)] = |deta| 2p " 
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and 

n+1 

Pn = —- 

When a = or 2, the representation I a {c) descends to a representation 
of the linear group Sp(2n, R). 

Define the map ip m : S{M p+q , n (R)) h-> C°°(G) by 

^ m {f){g) = (co(g)f)(0), f e 5(M P+?) „(1)), g e G. 

From the well-known formula of the oscillator representation u, we see 
that 

: <S(M p+9 , n (R)) ^ /"(a), 

where 

a = ~ 2 ~ ~ <0n ' an< ^ a = P — ? ( m °d 4). (3.2) 
The following is the fundamental result of Kudla-Rallis. 

Theorem 3.3. ([KRl]) The map ijj Piq induces a topological embedding 
with closed image: 

SF' q ^ I a (a) 

Remark: Analogous results hold for other classical groups. See |Zhj . 

4. Degenerate principal series: the transition 

coefficients 

Fix a maximal compact subgroup K± ~ U(n) of Sp(2n, M.) and let 
K be the inverse image of K\ in Sp(2n,IR), thus a maximal compact 
subgroup of Sp(2n,M). The Lie algebra of Sp(2n, M) has a Cartan 
decomposition 

sp(2n,M) = t ©p 

where £ is the Lie algebra of K\ and is isomorphic to u(n). 
Let 

A+ = {(Ai, A n ) e Z n : Ai > A 2 > ■ ■ ■ > AJ, 

and 

1 = (1,...,1)GA+. 

j 

Also let ej = ((L~~Cm, 0, 0), for 1 < j < n. 
We have the K-type decomposition: 
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where V 2X+s _i is an irreducible i^-module with highest weight 2A + 1 1, 
for A e A+. 

For /x = 2A + ~1, let be a K-type in / a (cr) and fix 7^ to the 
unique (up to a multiple) i^-highest weight vector in V^. We consider 
the tensor product pc <8> of lc modules, and the tc m &P 

mipcQVp — > I a (°) K 
m(p g) v) = p ■ v. 

For each 1 < j < n, there exists an element Xj in ZY(sp(2n, C)) (the 
universal enveloping algebra of the complexified Lie algebra) with the 
property that Xjfr^) is the image of the unique tc highest weight 
vector of weight /i + 2ej in p c (g> V^, valid for all \i. C.f. [L2\ Section 
3]. Thus Xjipf^) is a multiple of the unique tc highest weight vector 
7 M+2e:/ in V^ + 2e • • We call this multiple the transition coefficient from 
to V^ + 2e - Likewise there is a transition coefficient from to V^_ 2e . 
Note that the transition coefficients depend on the choice of 7^'s. 

By explicitly constructing an appropriate choice of tc highest weight 
vector 7^ for all /i (c.f. |L2l Section3]), the transition coefficients Aj(fjL) 
from Vfj, to V^+2ej, and the transition coefficients Aj(p) from to 
V^_ 2ej , are computed to be 

A+( f ,) = B+-2\ j , Aj( l i) = 2\ J -B-, (4.2) 

where 

B/ = -t7-p n -f Sr = (T-p n -f +J + 1. (4.3) 

Corollary 4.4. ( |KRlj ) I a {o~) is irreducible if and only if 

a + Pn + %^1. 

The explicit form of transition coefficients also allows us to determine 
the complementary series. We will follow the method used in |L2] . Each 
K-type V 2X+ a]_ of I a {o~) has a i^-invariant inner product given by 

</l,/2>A= / fl(k)h(k)dk. 

Jk 

Since V 2X+ a]_ is an irreducible i^-module, any i^-invariant inner prod- 
uct on V 2X+ ai is a multiple of (., .}\. Thus if (.,.) is a Sp(2n, R)- 
invariant inner product on I a {c) then there exists positive constants 
{c\} XeA + such that 

(/l,/ 2 )=C A (/ 1 ,/ 2 )A, yflJ 2 eV 2X+ a 1 . 
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Since the K-types of I a (u) are mutually orthogonal with respect (., .), 
(.,.) is completely determined by the constants {c\}. Using similar 
arguments as the U(n,n) case (see [Lit section 9]), we obtain the fol- 
lowing: 

Lemma 4.5. The inner product on I a {<j) defined by the constants 
{ca}a£A+ is Sp(2n, M) -invariant if and only if 

(-a -p n -\- 2Xj + j — l)c A+£j + {-a + p n + f + 2Xj -j + l)c A = 
for all A G and all 1 < j < n. 



We let 

-o - pn-%- 2\j + j - 1 C A 



-a + p n + | + 2Xj - j + 1 c x+ej 

Then I a {cr) is unitarizable if and only if N\ j < for all A G and 
for all j. 

Note that 

AT -a-(p n + f + 2A J -j + l) 



-^+(Pn+f + 2X j -j + l) 
We write ^ = p n + % + 2Xj - j + 1. Then N XJ = (-a - + 0- 

Thus N\j is real for all A and for j if and only if either Re(er) = or 
a is real. The case Re(cr) = corresponds to the unitary axis. If o is 
real, then 

iVA J = Z ^|<0^| ( x|<K|. 

The minimum value of |£| is \ if n + a is even, and if n + a is odd. 
This leads to the following 

Theorem 4.6. Ifn + a is even, then I a (cr) is unitarizable for \o~\ < |. 

5. SUBQUOTIENTS OF I a (cr) 

In this section, we shall give a detailed description of the module 
structure of / a (<r) when it is reducible. We shall describe all the irre- 
ducible constituents of I a (o-) and determine which of them are unitariz- 
able, i.e., possess a Sp(2n, M)-invariant positive definite inner product. 
We also describe the socle series and module diagram of I a (cr). 

Let 

a n + 1 + a . iN 

a = a + p n + - = a + . (5.1) 
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By Corollary 14.41 I a (<j) is irreducible if and only if a ^ Z. Thus we 
shall assume that a G Z throughout this section. Let fj, — 2A+(a/2)l n . 
Then by the results on transition coefficients in Section HI 



where 



2\ 3 ^ B+ and V„ -> ^_ 2e . 



2A, ^ B: 



n+/ \ n + a + 1 _ . 

BT(a, a) = -a +j-l=-a+j-l 



and 



n-/ \ n + a + 1 
R . (a, a) = a + j + 1 



a - m + a) + j. 



3 3 V"'-/ - 2 

Our method to determine module structure is as follows: 

(1) Represent by the point 2A G W 1 , where (xx,...,x n ) is the 
standard coordinates for IR n . 

(2) Barrier to block j-th rightward movement — > V^^ej at tj : 



B 7- 



(3) Barrier to block j-th leftward movement — > V^_2 ej at 



The barrier 



1 j 



Bf is effective if and only if it cuts at an 



"even" point, i.e. Bj G 2Z. The parity of Bj depends on the par- 
ity of a = a + and the parity of j. Similarly for the barrier 
£j , but the parity of B~ also depends on the parity of n + a. So our 
analysis will be divided into 4 cases. 



a\n + a 


odd 


even 


odd 


Case la 


CclSG 2cl 


even 


Case lb 


Case 2b 



We also define the "gap" between £t and ^ • by 
gap = Bf - Bj = -2a - 2. 



(5.2) 



5.1. Subquotients of I a (a): n + a odd. 



Case 1: n + a odd. 



Since a G Z and w+ ^ +1 G Z, we have a G Z. It follows from this and 
equation (15.21) that the gap Bf — Bj is even, i.e. Bf = Bj(mod 2). 
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This means that for each j, either both barriers and £~ are effective, 
or both are not effective. 



We consider two subcases: 
Case la: a = a+ ^±|±1 \ s Q dd 



Case lb: a = a + is even. 



For Case fa, We have 

B+ = -(a + n + ^ + 1 ) +j - 1 e 2Z j is even. 
Hence the effective barriers are £j and £ 2i for all 1 < i < no/2, where 



n = 2 



largest even integer less than or equal to n. (5.1.1) 



But for Case fb, We have 



b: 



n + a + 1 



) + j - 1 G 2Z j is odd. 



So the effective barriers are i^i-i an d ^-i f° r all f < i < (rii + f)/2, 
where 



ni = 2 



n+ 1 



1 = largest odd integer less than or equal to n. 

(5.1.2) 



Case la: a = o + 2±f±i is odd 

Let i and j be such that < i + j < n /2. We will define Rij(n, a, a) 
as follows: 



(i) For a < — f , let Rij(n, a, a) be the set of 2A such that 

2A 2l > B+ +2 > 2\ 2l+2 (5.1.3) 



2A no _ 2 j > B no _ 2j > 2\ 



n -2j+2 



(5.1.4) 



where 



n + a + 1 n + a — 1 
-o h 2? + f = -a V 2i, 



B n () -2j = 0- 



n + a + 1 . n + a — 1 

\-n -2j + f = a \-n - 2j. 
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(ii) For cr > 0, let Rij(n, a, a) be the set of 2 A such that 

2A 2l > B 2l > 2X 2i+2 (5.1.5) 

2A no _ 2 j > B no _ 2j+2 > 2A no _ 2 j+2 (5.1.6) 

where 

n + a + 1 n . „ n + a — 1 
B 2i =o g + 1 + = ° 2 + ' 



n + a + 1 n . . « n + a—1 



- B nn-27+2 = ~<? o + n - 2j + 1 = -<7 + 7i - 2j. 



Remarks: 

(a) When a = and i+ j = n /2, the two conditions (I5.1.5P and 
(I5.1.6P coincide. More precisely, 

R i:j (n,0,a) = {2A : 2A 2j > B 2i > 2X 2i+2 }. 

(b) The set Rij(n,a,a) defined above may be empty. When it is 
nonempty, it can be identified with the direct sum of all the K 
representations V 2X+ ^i such that 2A G Rij(n,a,a). 



Theorem 5.1.7. (Case la) Assume that n + a is odd, o is an integer 
and a = a + n+ " +1 is odd. 

(a) If a < — 1, then 



where 



T\ = max + a, 



In this case, the module diagram of I a (o~) can be obtained from 
Figure 1 by removing those R a b(n,o~,a) which are empty. In 
particular, the socle series of I a (a) is given by 



Soc'(/ Q (a)) 



© n < i+i < ri+1 -i Ruin, tr,a) l<l<J- ri , 



I a (a) />f-n + l. 



An irreducible constituent Rij(n,a,a) of I a (a) is unitarizable 
if and only if — ^ < a < — 1 and i + j = r±. 
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(b) If a = 0, then 

I a (0) = RijinAa) 

i+j=n /2 

is a direct sum of irreducible unitary submodules. 

(c) If a > 1, then 

J » = 0{^>^«): r 2 <i+j<^}, 
where 

r 2 = max ( y - a, Oj . 

In this case, the module diagram of I a (cr) can be obtained from 
Figure 2 by removing those R a b(n,a,a) which are empty. In 
particular, the socle series of I a (cr) is given by 



Soc'(/ q ((t)) 



0M_,+i< i+J <M Rij(n, a, a) 1 < I < f - r 2 , 
I a (a) l>m- r2 + i. 



An irreducible constituent Rij(n,a,a) of I a (cr) is unitarizable 
if and only if 1 < a < ^ and % + j = r 2 . 

Case lb: a = a + 2±f±l is even 

Let i and j be such that < i + j < (n\ + l)/2. We will define 
Rij{n, a, a) as follows: 

(i) For a < — 1, let Rijin, a, a) be the set of 2 A such that 

2A 2i _! > B+ +l > 2A 2i+1 (5.1.8) 

2A ni _ 2 j > B ni _ 2j > 2A ni _ 2g+2 (5.1.9) 



where 

n + a + 1 
~2 



= ~o + 2i, 



n + a + 1 n + a — 1 
5 m-2j = o- 5 +n 1 -2j + l = a + m - 2j. 
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(ii) For cr > 0, let Rij(n, a, a) be the set of 2 A such that 

2A 2i _i > ^ 2A 2m (5- 1 - 10 ) 
2A m _2j > -B ni _ 2j+2 > 2A ni _ 2 j+2 (5.1.11) 

where 

„ n + a + 1 
B 2 *-i =o o + 2 *> 



n + a + 1 „ . _ n + a — 1 



5 rt, -2i+2 = ~ a o h n x - 2j + 1 = -CT h Tli — 2j. 



Remark: When a = and i + j = (ni + l)/2, the two conditions 
05.1.81) and (I5.1.9P coincide, that is, 2A ni _2j = 2A(2i+2j-i)-2j = 2A 2 i_i, 

n + a — 1 . . n + a + 1 „ 
5-_ 2 ,+2 = j + ( 2 * + 2 ^- 1 )-^ = 2 + 2 ' = 

and 2A ni _2j+2 = 2A {2i+2 j-i)-2j+2 = 2X 2j+ i. Precisely, 

^•(n,0,«) = {2A : 2A 2i _ 1 > > 2A 2 ,:+i}. 



Theorem 5.1.12. (Case lb) j4ssnme thatn + a is odd, a is an integer 
and a = o + n+ 2 t+1 even. 

(a) If cr < — 1, then 

I» = 0{^(n ;( 7,c): ri <* + j<^±i}, 



ni + 1 

ri = max h cr, . 



where 



In this case, the module diagram of I a (a) can be obtained from 
Figure 1 by removing those R a b(n,o~,a) which are empty. In 
particular, the socle series of I a (a) is given by 

ri < i+i < ri+ ^ 1 J2tf(n, a, a) l<l<m±l- ri , 



Soc l {I a (a)) 



I a (a) />si+l_ ri + i. 



y4n irreducible constituent Rij(n,a,a) of I a (a) is unitarizable 
if and only if 

(i) — TjLk ^- < cr < — 1 and i + j — r 1 ; or 

(iij n zs odd, a e {0, 2} and (i, j) e { 0) , (0, 2±i) } . 
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(b) lfa = 0, then 



/«(0)= R t3 (n,0,a) 



is a direct sum of irreducible unitary submodules. 
(c) If a > 1, then 



where 



>i + l 
r 2 = max — a, . 



In t/izs case, £/ie module diagram of I a (a) can be obtained from 
Figure 2 by removing those R a b(n,a,a) which are empty. In 
particular, the socle series of I a (a) is given by 



Soc l (I a (a)) 



0iii+i_ m <i +i <!M_ti Rij(n, a, a) 1 < I < - r 2 , 
I a {(j) j>2i±l_ r2 + i. 



An irreducible constituent Rij(n,a,a) of I a (a) is unitarizable 
if and only if 

(i) 1 < a < and i+ j — r 2 ; or 

(ii) n is odd, a G {0, 2} and (i, j) e { (2±i, 0) , (0, 2±i) } . 



The module diagram of I a (a) for er < — 1 and er > 1 are given in 
Figure 1 and Figure 2 below. Here 

— = [— ] if a is odd, 

k={ ~ (5.1.13) 

ni + 1 r n + 1 . 

— ^— = [— ^— ] if a is even. 

and we write a constituent Rij(n, a, a) simply as Rij. 
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-Rfc,0 ^fc— 1,1 ' ^l,fc — i Ro,k 




^0,0 



Figure 1: Module diagram for I a (cr) (a < — 1) 



^0,0 




Rk,0 -R/c-l,! ... Rl,k-1 R 0,fc 



Figure 2: Module diagram for I a (cr) (a > 1) 

5.2. Subquotients of I a (a): n + a even. 



Case 2: n + a even. 

Recall that a = a + ™±|±I e i. So in this case, a + \ G Z, and 
consequently the gap between the barriers £t and £~ given by 

gap = —2(7 — 2 

is odd. It follows that along each coordinate axis, exactly one barrier 
is effective. More precisely, for each 1 < j < n, either is effective or 
£~ is effective, but not both. Again, we consider two subcases: 

Case 2a: a = a + 2±§±! is odd Case 2b: a = a + 2±f±i is even. 
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For Case 2a, is effective when j is even, and £~ is effective when j 
is odd. Case 2b is the other way round. 

Case 2a: a = a + 2±f±i is odd 

Since exactly one barrier is effective along each coordinate axis, the 
barrier partitions the K- types into 2 subsets. For 1 < r < + 1> 1^ 

X[ = {2A : 2A 2r _i < -B 2r _ 1 }, 

X£ = {2A : 2A 2r -i > ^2~r-i}' 

and for 1 < s < f + 1, 

1? = {2A : 2A 2j < B+}, 
Y 2 S = {2X:2X 2j >B+}. 

Now we let 

S(n) = : < i < ^±1, < j < ^}. (5.2.1) 

Then for (i, j) G S(n), we form the intersection 

"1+1 , -1 

L itj (n, a, a)= {x\ n • • • n x\ n xj +1 n • • • n x 1 2 ) n 

n I i 

(k, 1 n • • • n n n • • • n V ). 

More precisely, 2 A G Lij(n, a, a) if and only if 

2A 2i _! > B 2i _ x > 2A 2m and 2A 2j > £+ +2 > 2A 2j+2 , (5.2.2) 
where 

n + a + 1 „. _ _ n + ct _ 1 



= o- ^ + 2i - 1 + 1 = <t - + 2i - - 



and 



„4_ n + a + 1 n + a n . 1 

= -o- 2 + 2j + 2 - 1 = -a - + 2j + -. 

Note that L a<b (n,a,a) may be empty. If it is nonempty, then we shall 
identify it with the direct sum of all the K representations V 2X+SL \ with 

2A G L ajb (n,a,a). 

Remark. Let k = [n/2\. Then 

■ < % < k + 1, < j < k} n odd, 
S(n) = { (5.2.3) 
{(hj) '■ < i,j < k} n even. 



The following combinatorial result is elementary. 
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Lemma 5.2.4. (i) Assume that G S(n) and i — j < — 1. 

Then L it j ^ if and only if i — j > —& + |. In 'particular, if 
a < —1/2, then all such L it j is empty. 

(ii) For < i < k, L^j is always nonempty. 

(iii) // (j + l,j) G 5*(n) ; i/ien is always nonempty. 

(iv) Assume that G S'(n) and i — j > 2. Then L it j ^ if and 
only if i — j < —cr+ \ . In particular, if a > 1/2, then all such 
Lij is empty. 



Theorem 5.2.5. (Case 2a) Assume thatn + a is even, o- + | G Z, and 
g- = o- + S±S±1 is rfrf. 

(i) If a > 1/2, then 

'>)=0{^>,^«) : (i,j)eS(n), -1 < j - i < n}, 
u>/iere 

n =min (<r- §, [f]) , 
and eac/i L it j(n,a,a) which appears in the sum forms an irre- 
ducible constituent of I a (a). The module diagram of I a (a) can 
be obtained from Figure 3 by removing those L a ^{n, a, a) which 
are empty. In particular, the socle series of I a (a) is given by 

Q nr l (T a (rr) - / ®(*J)eS(n), -l<j-i<l-2 L i,j( n > °, a ) 1 < 1 < r l + 1, 

bOC [I [a) - \ I a (a) I > n + 2. 

^4n irreducible constituent L it j(n,a,a) of I a (cr) is unitarizable 
if and only if i = j + 1 or \ < a < [|] + \ and j — i — r±. 

(ii) If a < -1/2, then 

r(*) = G){I<i d (n,<T,a): (ij) G S(n), 0<i-j<r 2 }, 
where 

r 2 = min(-o- + \, [2±i]), 
and eaca Lij(n,a,a) which appears in the sum forms an irre- 
ducible constituent of I a (cr). The module diagram of I a (a) can 
be obtained from Figure 3 by removing those L a p{n, a, a) which 
are empty. In particular, the socle series of I a (a) is given by 

<Znr l (T a (n)) - / ®(»,j)eS(ri), r 2 -l+l<i-j<r 2 L i,j( n i °i a ) 1 < 1 < r 2, 

&0 1 1 )] ~ \ I a (a) I > r 2 + 1. 

An irreducible constituent L it j(n,a,a) of I a (cr) is unitarizable 
if and only ifi — j or — f 21 ^-] + \ < o~ < —\ and i — j = r 2 . 



18 DEGENERATE PRINCIPAL SERIES AND HOWE CORRESPONDENCE 



The module diagram of I a {cr) in the case when n + a is even and 
o + H+i+i i s odd can be obtained from the following rectangle (Figure 
3) by removing those L at b(n,<j,a) which are empty. Here k = [n/2] 
and we write a constituent L a ^(n, a, a) simply as L a ^. Note also that 
when n is even, the spaces L k+1 j (0 < j < k) are not defined. In this 
case, the rectangle below reduces to a square. 




Figure 3: Diagram for I a (a) when n + a is even and a + n +<^+ 1 ; s odd 

Case 2b: a + is even 

As in Case 2a, exactly one barrier is effective along each coordinate 
axis. But now is effective when j is odd, and £j is effective when j is 
even. So we have to define L a ^ b (n, a, a) differently. For 1 < r < ^^ + 1, 
let 

X[ = {2A : 2A 2r _ 1 < 

X r 2 = {2A : 2A 2r -i > ^2r- 1}> 
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and for 1 < s < f + 1, 

Y{ = {2\:2X 2s <B 2s }, 

Y 2 S = {2\:2\ 2s >B 2s }. 

We define the set S(n), as in Case 2a in equation (15.2.11) . Then for 
(i, j) G S(n), we form the intersection 

L itj {n, a, a)= (X\ D • • • D X\ D n • ■ ■ H X x 2 + ) n 

(K, 1 n ■ ■ • n Y 2 j n n • • • n 

More precisely, 2 A e L^{a, a) if and only if 

2A 2i _x > > 2A 2m and 2A 2j > B 2j > 2X 2j+2 . (5.2.6) 

where 

-BJ , i = —o h 2i + 1 — 1 = —a h 2i 

21+1 2 2 2 

and 

ra+a+l ra+a 1 

B 2J = o + 2 j + 1 = a - — + 2 j + - . 

Note that L a ^(n, a, a) may be empty. If it is nonempty, then we shall 
identify it with the direct sum of all the ^-representations V 2X+ a_i with 
2A e L ajb (n,a,a). 



Theorem 5.2.7. (Case 2b) Assume thatn + a is even, a + | 6 Z, and 
5" = a + is even. 

(i) If a > 1/2, then 

I a (v) = Q){L hj (n,a,a): (i,j)eS(n), 0<i-j<r 2 }, 

where 

r 2 = min(a+|, [^±±]), 

and each Lij(n,o~,a) which appears in the sum forms an irre- 
ducible constituent of I a (a). The module diagram of I a (a) can 
be obtained from Figure 4 by removing those L ajb (n, a, a) which 
are empty. In particular, the socle series of I a (a) is given by 

{ ®(i,j)eS(n),0<i-j<l-l L iA U ' <r, «) 1 < 1 < r 2> 
Soc ; (r(a) = ! 

{ I a (a) I >r 2 + l. 

An irreducible constituent Lij(n,a,a) of I a (a) is unitarizable 
if and only if i = j or | < a < [^rp] — \ and i — j = r 2 . 
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(ii) If a < -1/2, then 



J » = 0{^>,^a): (i,j)eS(n), -l<j-i<n}, 



where 



n = min(-<7- |, [§]) , 



and eaca L it j(n,a,a) which appears in the sum forms an irre- 
ducible constituent of I a (a). The module diagram of I a (a) can 
be obtained from Figure 4 by removing those L ajb (n, a, a) which 
are empty. In particular, the socle series of I a (cr) is given by 



Soc l (I a {a)) 



©(iJ)eS(n), ri-J+l<i-i<ri L iA n > ^ «) 1 < * < ^1 + 1, 

I a (a) l>n + 2. 



An irreducible constituent L it j(n,a,a) of I a (a) is unitarizable 
if and only if i = j + 1 or — [|] — | < (J < — | and j — % = r-y. 



Finally, we describe the module diagram of I a (o-) in the case when 
both n+a and a+ n+< ^ +1 are even. It can be obtained from the following 
rectangle (Figure 4) by removing those L a b(n,a,a) which are empty. 
As before, k = [n/2], and we write a constituent L a ^{n, a, a) simply as 
L a b . When n is even, L k+1 j (0 < j < k) is empty so that this rectangle 
reduces to a square. 
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-'k+1,0 



Lk+l 



^fe-1,0 



->k,l 



-'fc+l.fc-l 



Lift L2,i 
\ / \ / 



Lk,k-i 
/ \ 

\ / 
Lk-i,k 



Lk+i,k 



Lo,k- 



Figure 4: Diagram for I a (a) when both n + a and cr + n +<^+ 1 are even 

6. The image of tp p>q in J a (o-) 
In this section, we will describe the image of ip Ptq in I a (a): 

where 

P+Q n+1 
a = — — , and a = p — q (mod 4). 

We allow (p,q) = (0,0), in which case we understand fi 0,0 to be the 
trivial representation of G — Sp(2n, K). 
Denote m = p + q. Observe that 

n + a = n + m (mod 2), 

and 

n + a + 1 m + a , . 
a = a+ = — - — = p (mod 2). 

Recall that our analysis of I a ((r) is divided into 4 cases: 
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a\n + a 


odd 


even 


odd 


CclSG lcl 


CcLSG 2cL 


even 


Case lb 


Case 2b 


our analysis of Q p,q will also be 


p\n + m 


odd 


even 


odd 


CcLSG XcL 


CctSG 2cL 


even 


Case lb 


Case 2b 



Throughout this section, we assume that we are at a point of re- 
ducibility, namely 

n + a + 1 _ 
cr H G Z. 



If 



p+q— (n+1) 



a and p~q = a (mod 4), we say that we have a possible 
embedding of Q p,q (into I a (a)). 

We shall give the detailed description of Q p,q in I a {cr) in the following 
two subsections. We end this section with a corollary which gives 
conceptual underpinning to our results. 

Theorem 6.1. The relationship between Q p,q 's and / a (cr) is as follows. 

(a) —p n < a < 0: The irreducible submodules of I a (a) are given by 
the possible embeddings of Q p ' q 's, and all of them are unitary. 

(b) Unitary axis (a = 0) (when n + a is odd): 



r(o) 



e 



n p > q . 



p+q=n+l 
p—q=a (mod 4) 



(c) a > 0: The reducibilities of I a (a) are completely accounted for 
by the possible embeddings of Q p ' q 's. 
- See Part b) of Theorems EXj [Oj EJJ and EjO for 
the precise statements. 

6.1. The image of ip Ptq in / a (cr): m and n different parity. 



Case 1: n + m odd. 



We consider two subcases: 

Case la: p is odd 
Case la: we write 

p = 2i + l, q = 2j + e, 



Case lb: p is even. 
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so that p + q = m. Here 

1 m even, 



Note that 



i + j 



m odd. 
m — 1 — e m + n — (n + 1) n 



+ <T. 



Case lb: we write 



p = 2i, q = 2j + e', 



so that p + q = m. Here 



m even, 

1 m odd. 



Note that 



m — e' m + (m +1) — (n + 1) n\ + 1 

i + j = — = — = -r- — = — ^ f7. 



2 2 
In accordance with I5.1.13[ we let 

no _ n 
2 ~ [ 2 J 



k 



ni + 1 r ri + 1. 



if p is odd, 



if p is even. 



2 2 

Recall that we have the following module diagrams: 



Case 1: Module diagram for I a {u) (—k < it < 0); n = fc + a 



n.fe— 1 rt o,fc 



Case 1: Module diagram for I a (<T) (1 < cr < A;); r2 = A; — tr 
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We introduce one notation. For an irreducible submodule R s>t of 
I a (<j) where a > 0, denote by -< R Sjt y the submodule of I a (cr) gener- 
ated by R s j. 

-4 R st y= (B{R ltJ :i>s,j> t}. (6.1.1) 

Recall that 

m - (n + 1) . . 

<T = , and p — q = a (mod 4). 



Theorem 6.1.2. (Case la) Assume thatn + m is odd, and p is odd. 
(a) If a < 0, then 



q—t ■ 

2 ' 2 



(b) Ifa>\, then 

=~< R s>t y, 

where s = max (0, ^) , and t = max (0, n+1 ~ e ~ p ) . 
Remarks: 

(a) For = (2=^, ^), we have noted that i + j = 1 f + o- = r 1 . 
Thus is at the bottom layer of the module diagram of I a (cr), 
namely is an irreducible submodule. The collection of Rij's 
exhausts the set of all irreducible submodules of I a {o~). 

(b) If q < n, and p < n + 1 - e, then s = max (0, ^=2) = "=2, t = 
max (0, »±l=£=g ) = n+1 ~ £ - p , and s + t = ^ -a = f -a = r 2 . 
Such i? Sit 's are exactly those at the top layer of the module 
diagram of I a {c), namely irreducible quotients. The rest of 

t's are those on the "left boundary" (s = 0) or the "right 
boundary" (t — 0). 



Theorem 6.1.3. (Case lb) Assume thatn + m is odd, and p is even. 
(a) lfa<0, then 

2 ,_ 2~ 



(b) Ifa>l, then 

=^ R. t y, 



where s = max (0, ^±±=2) , and t = max ((D, " +1 ~ e ~ p ) . 
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Remarks: 

(a) For = (|, SL ^-), we have noted that i+j — + a = r 1 . 
Thus Ri j is at the bottom layer of the module diagram of I a {cr), 
namely is an irreducible submodule. The collection of R^/s 
exhausts the set of all irreducible submodules of I a (<y)- 

(b) If q < n+1, andp < n+l-e', then s = max (0, = 

t = max (o, n+1 7'~ p ) = n+1 -/- p , and s + t = 2±f^ -a = 

^j^- — a = r 2 . Such R S) tS are exactly those at the top layer 
of the module diagram of I a {u), namely irreducible quotients. 
The rest of R s /s are those on the "left boundary" (s = 0) or 
the "right boundary" (t — 0). 



6.2. The image of ip p ^ q in I a (a): m and n same parity. 

Case 2: n + m even. 

We consider two subcases: 

Case 2a: p is odd Case 2b: p is even. 

We introduce one similar notation in this case. For an irreducible 
submodule L Sjt of I a (cr) where a > 0, denote by -< L s t y the submod- 
ule of I a (cr) generated by L s>t : 

<L y= \ ®{Li,j -i>s,j <t}, (Case 2a), (Q 2 1) 

s,t \®{Li :j :i<s,j >t}, (Case 2b). 

As in Subsection 15.21 we let k = [n/2]. First recall the module 
diagram of I a (cr) for Case 2a: 

Lk,k L k -i,k-i .... Li t i Lq.o 



Lk,h-1 L2, 



->k+l,k+l-r 2 ■■■ L r 2 ,0 



Case 2a: Diagram for I a (o) when n + m is even and p is odd; <r < — i; r2 = — <r + i < [ "j" ] 
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Lk + l,k £fc,fc-l £2,1 £l,0 

Case 2a: Diagram for I a (a) when n + m is even and p is odd; cr > i; ri = ct — ^ < [||] 

Theorem 6.2.2. (Case 2a) Assume that n + m is even, and p is odd. 

(a) If a < —\, then 

VL P ' q = L n+l-q p-1 . 

2 ' 2 

(b) Ifa>\, then 

W« =^ L,, t y, 

where s = max (0, n+ \~ q ), and t = min ([f], 
Remarks: 

(a) For (i, j) = (=^=2, 2fi), we have i- j = = -<r+± = r 2 . 
Thus Li j is at the bottom layer of the module diagram of I a (cr), 
namely is an irreducible submodule. The collection of L^fs 
exhausts the set of all irreducible sub modules of I a {o~). 

(b) If q < n + 1, and £=± < [f ], then s = max (0, *±±=2) = *±±=l, 
t = min ([f ], 2fi) = 2fi, and t-s = m ~ ( 2 n+2) = a-± = n. Such 
I/ Sj t's are exactly those at the top layer of the module diagram of 
I a {<y), namely irreducible quotients. The rest of L S) t's are those 
on the "left boundary" (t = k = [|]) or the "right boundary" 
(s = 0). Note that when n is even, the submodule is 
empty. 



Next recall the module diagram of I a (cr) for Case 2b: 
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Lift 1/2,1 .... £fc,k-l £fc+l,fc 




Case 2b: Diagram for I a (cr) when n + m and p are even; ct < — i; n = — a — i < [2] 



'i-2,0 ■■■ -Lfc-|_i,fc4-i_ r2 




^0,0 £fc-l,k-l ifc.fc 



Case 2b: Diagram for I a (a) when both n + m and p are even; cr>i;r2=cr+|< t 21 ^] 

Theorem 6.2.3. (Case 2b) Assume thatn + m is even, and p is even. 

(a) If a < —\, then 

QPtQ —— p n — q . 

2 ' 2 

(b) Ifa>\, then 

=^ L a>t y, 

where s = min §) , and t = max (0, . 

Remarks: 

(a) For (i, j) = (|, ==2), we have j - % = = -a - \ = n. Thus 
Ljj is at the bottom layer of the module diagram of / a (o"), 
namely is an irreducible submodule. The collection of L^s 
exhausts the set of all irreducible submodules of I a (cr). 
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(b) If f < [2±1] and q < n, then s = min ([S±l], f ) = f, t = 
max (0, 2f2) = ==2, and s - t = = a + \ = r 2 . Such 
L s /s are exactly those at the top layer of the module diagram 
of I a {cr), namely irreducible quotients. The rest of L S) t's are 
those on the "left boundary" (t — 0) or the "right boundary" 
(s = Note that when n is even, the subquotient Lk+ij 

is empty. 
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